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ABSTRACT 


NASA’s vision for space exploration (February 2004) calls for development of a new 
crew exploration vehicle, sustained lunar operations, and human exploration of Mars. To meet 
the challenges of planned sustained operations as well as the limited communications between 
Earth and the crew (e.g., Mars exploration), many systems will require crews to operate in an 
autonomous environment. It has been estimated that once every 2.4 years a major medical issue 
will occur while in space. NASA’s future travels, especially to Mars, will begin to push this 
timeframe. Therefore, now is the time for investigating technologies and systems that will 
support crews in these environments. Therefore, this summer two studies were conducted to 
evaluate the technology and systems that may be used by crews in future missions. 

The first study evaluated three commercial Indoor Positioning Systems (IPS) (Versus, 
Ekahau, and Radianse) that can track equipment and people within a facility. While similar to 
Global Positioning Systems (GPS), the specific technology used is different. Several conclusions 
can be drawn from the evaluation conducted, but in summary it is clear that none of the systems 
provides a complete solution in meeting the tracking and technology integration requirements of 
NASA. From a functional performance (e.g., system meets user needs) evaluation perspective, 
Versus performed fairly well on all performance measures as compared to Ekahau and Radianse. 
However, the system only provides tracking at the room level. Thus, Versus does not provide the 
level of fidelity required for tracking assets or people for NASA requirements. From an 
engineering implementation perspective, Ekahau is far simpler to implement that the other two 
systems because of its wi-fi design (e.g., no required runs of cable). By looking at these two 
perspectives, one finds there was no clear system that met NASA requirements. Thus it would be 
premature to suggest that any of these systems are ready for implementation and further study is 
required. 

The second study evaluated current medical packs, used on-board the International Space 
Station (ISS), in the execution of an emergency medical procedure as compared to a modified 
design. An experiment using 13 participants found no difference in performance time between 
the two packs; however, it did find a marginally significant difference (p = 0.08) in the number 
of errors with the modified design resulting in less errors. Using the experimental data collected, 
a computer model was developed that allowed for running larger sample sizes. Results from this 
model found a statistically significant differences for time and errors (p<0.05). Further modeling 
evaluated the effect of errors on performance time. Results once again found a statistically 
significant difference for time and found that the current pack design’s performance in time was 
4 times greater when errors were considered as compared to the design when errors were 
ignored. However, the modified pack only saw a 2 times increase when errors were considered. 
Given that NASA typically is dealing with small samples and limited resources to test 
participants, modeling should be considered to evaluate designs prior to experimentation. Future 
work will investigate the value in developing a desktop application for modeling medical 
procedures independent of experimentation. This modeling does not preclude experimental 
efforts, but it does provide guidance in conducting experiments. 

Both studies highlight issues that require further investigation. These studies are just one 
step needed to prepare systems and technologies for future planned human exploration. 
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INTRODUCTION 


Two studies were conducted as a part of this report. The first study evaluated Indoor 
Positioning System (IPS) technology for use on-board the International Space Station (ISS) as 
well as any future NASA exploration vehicles. The second study evaluated current medical 
packs used on board the ISS and explored the use of modeling techniques for simulating human 
experiments. 

STUDY 1: LOCATION TRACKING STUDY 

IPSs are being used to track equipment and people in many different settings 
commercially including hospitals, university libraries, and museums. While similar to Global 
Positioning System (GPS) tracking devices, the specific technology used is different. IPSs are 
functionally in-door equivalents to GPS tracking systems. 

The Biomedical Systems Division in the JSC Engineering Directorate was tasked to 
evaluate the usability of three commercial IPSs that are currently being used by several 
industries. Biomedical Systems Division personnel requested the Usability Testing and Analysis 
Facility (UTAF) to provide a human factors assessment of the three systems, Ekahau, Versus, 
and Radiance. All three systems had been procured as evaluation systems. This study report 
summarizes the UTAF human factors assessment findings, including: a general description of 
each system, the human factors assessment approach employed, identified issues and 
recommendations, and a plan for possible future work. 

Location Tracking System Descriptions 

Three commercial systems were evaluated as a part of this assessment. A brief overview 
of each of the systems evaluated is provided. 

Ekahau: The Ekahau Positioning system is developed by Ekahau, Inc. of Saratoga, CA. 
The Ekahau Site Calibration™ method is used for collecting radio network sample points from 
different site locations. Each sample point contains received signal strength intensity (RSSI) and 
the related map coordinates, stored in an area-specific positioning model for accurate tracking. 

Ekahau advertises up to a 1 meter (3*/2 ft) average positioning accuracy based on the 
positioning model technology. Ekahau allows an administrator to define “logical” areas that can 
be used to identify an individual’s location. These logical locations could correspond to a room 
or a portion of a room. Ekahau's positioning and site survey technologies work with all industry- 
standard wi-fi (IEEE 802.11a/b/g) access points and most network cards without proprietary 
hardware. Therefore, Ekahau when implemented can take advantage of existing wi-fi networks. 

Versus: Versus Information System (VIS) is developed by Versus Technologies, Inc. of 
Traverse City, MI. Versus is a combination radio frequency (RF) and infrared (IR) system that 
can identify at “room-level” the location of an individual or piece of equipment. Versus has two 
types of badges: those whose purpose is solely to monitor an individual’s location, and those that 
allow an individual to send an alert to an operator console. 

Radianse: Radianse, Inc., formerly Sentinel Wireless, Inc., is headquartered in 
Lawrence, Massachusetts. Radianse has developed an active -Radio Frequency Identification 
(RFID) technology that provides identification. Radianse receivers use standard Ethernet wiring 
and connect directly to a network where they require miniscule bandwidth. Internet Protocol (IP) 
addresses can be either Dynamic Host Configuration Protocol (DHCP) or static IP. Each receiver 
covers up to a 60-foot diameter. Location data from Radianse badges are collected by Internet 
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protocol-based Radianse receivers, and transmitted over any existing local area networks (LAN) 
to Radianse Location Software. Radianse location data can be shared with other systems and 
databases using standards such as Open Database Connectivity (ODBC), Extensible Markup 
Language (XML), Short Messaging System (SMS), Java, and JavaScript. Radianse badges 
contain two programmable buttons that can be set to send different messages to the operator 
console. 


HUMAN FACTORS ASSESSMENT APPROACH 


Participants 

The UTAF requested 10 (4 male, 6 female) different individuals from the Habitability 
and Human Factors Office (JSC-SF3) and the Biomedical Systems Division (JSC-EB) to serve 
as evaluators of the system. A total of five participants were used for the individual tests as 
described below in the procedure section and six pairs of individuals were used for the team 
evaluation described in the procedure. Individuals participated in one or more of the tests. Since 
the systems and not the participants were being tested, participants served only as a means of 
moving the transmitters through the facility. No crew members were used for this evaluation. 

Test Facility and Materials 

This study was conducted in the Advanced Integration Matrix (AIM) habitat located at 
NASA’s Johnson Space Center, building 29. Three areas in the module were used including an 
area with two floors. Transmitters were carried or worn by participants in this study depending 
on the location system, Participants carried a HP iPaq Pocket PC Personal Digital Assistant 
(PDA) with a compact flash wi-fi card for the Ekahau system. Participants wore a battery 
operated RF badge for the Radianse system. Participants wore the Versus Personnel Alert badge, 
a battery operated RF/IR badge, which allows for monitoring and sending alerts. We only tested 
the monitoring function. 

A dedicated laptop computer was used to monitor the personnel traversing the AIM 
facility and also used to capture the location data. This laptop was configured with each system’s 
software as provided by three vendors in their evaluation packages. Each system was configured 
by an in-house engineer trained on the systems from JSC-EB. Prior to any testing, all targets 
were evaluated by placing the particular transmitter on predefined targets in the same orientation 
to ensure the system would recognize the transmitter independent of people transporting the 
transmitter. 

Existing video cameras located throughout the AIM facility were used to capture the 
participants traversing the facility on VHS tapes. In addition, the existing intercom system was 
used to tell participants when to travel to the next target in the path. 

Procedure 

Participants were given an overview of the testing being conducted, signed a consent 
form approving video recording of the session, completed a demographic survey, and received a 
safety briefing so as to avoid the potential tripping hazards within the AIM facility. In addition, 
participants completed a walkthrough of path they were to follow so that they were aware of the 
targets in the facility. Once the system was enabled, the participant was either given the PDA to 
carry or had the receiver attached to their clothing. In either case, all participants canted the PDA 
or wore the receiver in the same position. These individuals traversed the AIM facility while 
wearing/holding the transmitters of each of the systems. 
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Five conditions were tested for this study and the exact procedure followed for each 
condition is described below. 

Condition 1: Individual walking (Individual) 

Participant Procedure : A single participant, holding/wearing a transmitter, walked to 
each of the sequentially numbered targets of the red path (refer to Figure 1) when they were 
instructed to do so (i.e., test conductor announced “Next” over the intercom system). Participants 
were told when to begin walking to the first target (target 2) based on the computer clock used by 
the systems to time stamp their location. The participant would stop at each target for a period of 
approximately 15 seconds, and advance to the next target based on a verbal command given 
through the intercom system. The participant continued the walk through all 15 targets. A total of 
five different participants were used for this condition. 

Condition 2: Two-person side-by-side walking (Pairs) 

Participant Procedure : Two participants walked the identified red path together. 
Participants advanced through the targets as in condition 1. A total of six participants (3 teams) 
were used for this condition. 

Condition 3: Two-person opposite walking (Opposite) 

Participant Procedure : Two participants walked the identified paths (blue and green, 
refer to Figure 1). One participant started on the blue path and one participant started on the 
green path. The participants passed each other on the paths. Participants advanced through the 
targets as in condition 1. In the module with two floors, participants first walked in parallel along 
the module (e.g., one on the 1st floor and one on the 2nd floor) in the same direction. On a 
second pass through the module, participants began at different ends on different levels and 
crossed over one another in the middle. This required participants to pass one another, but on 
different levels. A total of six participants (3 teams) were used for this condition. 

Condition 4: Hidden transmitter detection (Individual) 

Participant Procedure: A participant was asked to walk to predefined locations for each 
test location. In this evaluation, transmitter (PDA or badge) was hidden in different clothing and 
a computer bag to determine if it could still be detected by the system. One participant at several 
randomly selected locations was used. 

Condition 5: Obstacle transmitter detection (Individual) 

Participant Procedure : A participant was asked to walk to predefined locations for each 
test location. For this condition, several obstacles were selected for testing and these obstacles 
were placed in path of receivers. Locations were randomly chosen and obstacles included metal, 
plastic, wood, Plexiglas, and cardboard. One participant at several randomly selected obstructed 
locations. 
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Route Maps 



FIGURE 1: MAPS OF PATHS WALKED AND EXAMPLE NUMBERING (PHOTO) 
Measures of Performance 

Three different measures of performance were identified for evaluation of the three 
systems: errors, time to detect, and percentage time correct. Each measure is described in Table 
1 . 


TABLE 1: MEASURES OF PERFORMANCE 


Measure 

Description of Measure 

Number of Errors 

An error could be of two types: fail to detect (Miss) or detection of the 
participant in one location when they were actually in another location (False 
Alarms). An error occurred whenever the system failed to detect the person 
correctly while standing at a target for a period of 15 seconds or failed to detect 
them upon arrival at a new target. This method of counting error follows the 
Signal Detection Theory (SDT) method as developed by Green and Swets 
(1966). 

Time to Detect 

This measure evaluates the time it took for the system to identify the individual 
at a target. Four methods of time to detect were defined: (1) early (-) and late 
detection (+) time were counted, these times could potentially cancel each 
other out; (2) absolute value time to detect (e.g., early or late detection was 
counted); (3) all times late or early, within +/- 2 seconds were dropped and 
average time calculated; (4) late detection only time was counted. 

Percentage Time Correct 
(used as replacement for time to 
detect measure as will be 
discussed in report) 

This measure is the total number of seconds the system properly detected the 
participant compared to the total number of seconds for which the participant is 
standing at the path targets. Time was only counted for the time the participant 
was at an actual target. No attempts were made to assess the accuracy of the 
system while the participant was traversing the path. Using the SDT model 
above, this method assessed the percentage time of hits compared to total time 
available for a hit. 


Limitations of the Study 

Before discussing the results of the study, it is important to understand the limitations of 
this study up front. 
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• The capabilities of the systems were not equivalent. For example, Ekahau provided a display 
showing a map of the AIM facility and the current location of the individual; however, the 
other two evaluation systems tested did not have this capability. 

• The systems were set up and calibrated by a trained individual on the EB team, rather than 
the respective vendors. It is possible that with vendor support of the set up, each system 
might have performed better. 

• The four wi-fi routers used by Ekahau in this study included NetGear (1), Linksys (1), and D- 
Link (2). There were some indications that accuracy may have differed for the different 
vendor routers. This study did not evaluate this possibility. 

• The Ekahau system periodically indicates fluctuations (1 to 2 second) that will place an 
individual in the wrong location. As a result, our analysis looks at the impact of these 
fluctuations on system performance. There was some discussion that these anomalies might 
be due to the wi-fi points, settings within the software, or versions of the software. This study 
did not evaluate all of these factors, and thus they should be considered in future evaluations. 

RESULTS 

The results will be discussed by looking at each of the measures discussed in Table 1. 
Number of Errors 

Figure 2 displays the errors per system. Ekahau was prone to minor fluctuations (1 -2 
seconds) in its detection of a participant. The system would sometimes flip between areas when 
the participant was actually standing still on a target. Because of these fluctuations, the initial 
analysis of Ekahau showed an error rate of over 100% when each fluctuation was counted as an 
error since the number of errors exceeded the number of targets. Consulting with JSC-EB and 
review of the fluctuations found that the fluctuations were minor. Therefore, it was decided that 
Ekahau would be evaluated by eliminating any of the minor fluctuations (e.g., 1-2 seconds). 

Thus one will notice in Figure 2 that the errors displayed for Ekahau display a NF or no 
fluctuations indicator. In addition, since Ekahau can detect down to a logical area and the other 
two systems, as tested, could only detect to room level, Figure 2 displays the error rate for 
Ekahau at both the logical and room level. Figure 2 also shows the error rate for conditions 1 
(Individual), 2 (Pairs), and 3 (Opposite) for all systems. 

As one can see in Figure 2, Versus had the lowest error rate among the systems tested and 
Ekahau by logical area had the highest error rate. When Ekahau is evaluated at room level in 
comparison to the other two systems, its error rate decreases substantially; however it is still 
quite a bit larger than the other two systems. Even with Versus having the lowest error rate, its 
rate is still approximately 14%. 

To assess the severity of the errors received, each of the errors was classified into three 
categories: (1) Adjacent - identified the individual in an adjacent room to their actual location; 

(2) Nonadjacent - identified the individual in an area non adjacent to their actual location; and, 

(3) Different floor - identified the individual on a floor different that where they were located. 
Because of the uniqueness of the AIM facility compared to facilities in which these systems are 
typically implemented - for example, the open second floor and total metal structure, we felt it 
was important to understand the types of errors occurring. This may be useful to vendors of the 
systems as well as to JSC. Table 2 provides a percentage breakdown of the types of errors 
encountered. 
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% Errors 



Figure 2. Percentage of error by system and by condition 


TABLE 2: PERCENTAGE OF TY 

PES OF ERRORS 

System 

Adjacent 

Nonadjacent, Same 
Floor 

Different Floor 

Nonadjacent, 
Different Floor 

Ekahau 

42.70% 


53.93% 

3.37% 

Radianse 

54.17% 

9.72% 

19.44% 

16.67% 

Versus 

63.64% 

0.00% 

13.64% 

22.73% 


Time to Detect 

Initially the time to detect measure was identified as a system measure of performance. 
However, because of the fluctuations in the Ekahau system, it was felt that this measure would 
be difficult to assess fairly. Therefore, this measure was replaced by the percentage time correct 
measure. 

Percentage Time Correct 

Figure 3 displays the percentage time correct on target for each system. Once again, 
Ekahau was evaluated several different ways because of the differences between it and the other 
systems. Ekahau was first evaluated including any and all fluctuations in a location and then 
excluding all small fluctuations (1-2 sec). Once again, it was also evaluated at the logical and 
room level. 

Versus once again had the largest percentage correct, 96.5%, as compared to the other 
systems. Ekahau including fluctuations at the logical area had the lowest percentage time correct, 
41.45%. The percentage time correct and number of errors in detection follow similar patterns of 
performance except in the opposite direction. Thus, the higher the numbers of errors, the smaller 
the time percentage correct as confirmed by a Pearson’s correlation of 0.98. 
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% Correct Time on Target 



Figure 3: Percentage of time a system detected correctly by system and by condition 
Hidden Transmitter Detection 

For this evaluation, a single subject was used for the evaluation. The transmitter for each 
system was placed in different potential items that could hide the signal. The five tested 
scenarios were: (1) in the pocket of a coat (light colored); (2) under a light colored shirt; (3) in a 
computer bag; (4) in pant (jeans) pocket; and (5) under a dark colored shirt. Ekahau and 
Radianse were found in all hidden locations. Versus was found in all locations with the exception 
of three locations: (1) in a computer bag; (2) in pant (jeans) pocket; and (3) under a dark colored 
shirt. This finding was expected since the Versus system requires a good 1R signal to detect the 
badge. 

Obstacle Transmitter Detection 

Several different obstacles were evaluated to determine if the transmitters would be 
impacted including Styrofoam, Plexiglas partitions, metal panels, and a large piece of equipment. 
In addition, we asked the participant to enter areas with the transmitter away from the receiver. 
Groups of people also were used for evaluation. Each of the systems were discovered in all 
locations with some minor exceptions. Versus could be found in all locations except when the 
participant entered an area with their back turned or when they stood behind a large piece of 
equipment. Ekahau experienced fluctuations is signal when a group of four people were in front 
of the receiver. Ekahau would locate the individual in the wrong location. 

DISCUSSION 

As this was a preliminary study, only a small sample of participants was used for the 
study and no statistical analysis was conducted. However, this study did allow for testing of the 
study protocol and the collection of some initial data for assessment of the three systems. The 
small sample size of participants is a limitation; however, it did provide a very good assessment 
of the capability of the systems as they stand today in the AIM facility. 

When all three systems are compared on their ability to identify an individual’s location 
within a small area, Ekahau was the only system tested that provided this capability. The other 
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